Multi-label Active Learning Based on Maximum Correntropy Criterion: Towards Robust and Discriminative Labeling

Multi-label learning is a challenge problem in computer vision fields. Since annotating a multilabel instance costs greatly, multi-label classification has become a hot topic research. State-of-theart active learning methods either annotate all the relevant samples without diagnosing discriminative information in the labels or annotate only limited discriminative samples manually, that has weak immunity for the outlier labels. In this study, we focus on the outlier labels to reduce the label information redundancy and the number of labeled instances. A multi-label active learning based on Maximum Correntropy Criterion (MCC) is proposed, which is robust when the outlier happens. Figure 1. The influence of outlier label in the learning process. Figure 3. The influence of the outlier labels for the measurement of similarity MOTIVATION To understand our motivation directly, Figure. 1 shows the influence of the outlier label to learn a model for multi-label classification. Since the object that corresponds to the outlier label covers the less regions in the image than the most relevant objects., the features of the outlier label are also not clear in the features of the image. Hence, with such a feature to train a model, it is hard to learn the object in the new image which is a outlier label in the training images. A classic example is shown in Figure. 2 to show the outlier label. Figure 2. The interface of two properties for outlier labels. Left: The outlier label (Lion) is relevant to the image; right: the outlier (Lion) is much less relevant to the image than the most relevant label (Tree) is Denote and as the selected instance and two relevant labels, respectively. Define as the outlier label, if it has two properties. The first one is that is a relevant label to the instance , and the second is that is much less relevant to than is. Figure. 2 shows the two properties. The definition of the outlier label is consistent with the fact that, given an image, some labels’ relevance to it is apparent, which can be recognized at first glance by the oracle, and some labels’ relevance is veiled, which may need much effort for the oracle to label. The definition of outlier label is also consistent with the query types proposed in [1]. In active learning, representative information is important to select the most informative samples[2]. However, it is hard to measure the similarity between the multi-label instances with features when the outlier labels. Figure.3 shows how the outlier labels are influent to the similarity measurement. To overcome the influence of the outlier labels, we use the MCC to measure the uncertainty and representatveness in multi-label active learning with both label information and features information. • To the best of our knowledge, it is the first work to focus on the outlier labels in multi-label active learning based on MCC. • A an approach is derived to make the uncertain information more precise with the prediction labels of the unlabeled data . • The proposed representative measurement considers labels similarity by MCC. A new way is provided to merge representativeness into uncertainty. 2. Uncertainty Measured by MCC Minimum Margin is a popular approach to measure the uncertainty [4, 5]. We extend it to multilabel learning with MCC loss function, the objective function is as follows: 3. Representativeness Measured by MCC Since the similarity measurement based on the features is not confident in multi-label learning, a similarity that combines both features and labels is adopted. H H With the similarity measurement, the representative information for each sample is as follows: Then the representative part can be presented with the whole unlabeled data as follows: 4. The Objective Function To utilize the uncertainty and representativeness, we combine the two parts with a tradeoff parameter, and the objective function is as follows: RESULTS Compared with state-of-the-art methods on 12 multi-label data sets 1. Maximum Correntropy Criterion Correntropy is a similarity measure between two arbitrary random variables a and b, defined by where Kσ(•) is the kernel function that satisfies Mercer theory and E[•] is the expectation operator. With such a definition, the properties of correntropy are symmetric, positive and bounded. Since the joint probability density function of a and b in practice is unknown, and the available data are usually finite, the sample estimator of correntropy is usually adopted by where the kernel function is the Gaussian kernel. According to [3], the correntropy between a and b is given by The objective function is called maximum correntropy criterion (MCC). THE PROPOSED APPROACHCONCLUSIONContributions Limits• A mechanism should be developed to select the tradeoff parameters adaptively and make theproposed method more practical.REFERENCES[1] S.-J. Huang, S. Chen, and Z.-H. Zhou: Multi-label active learning: query type matters. in :IJCAI,pp: 946–952(2015).[2] R. Chattopadhyay, Z. Wang, W. Fan, I. Davidson, S. Panchanathan, and J. Ye. Batch mode activesampling based on marginal probability distribution matching. in: KDD, pp: 741-749, (2012)[3] R. He, W.-S. Zheng, T. Tan, and Z. Sun: Half-quadratic-based iterative minimization for robustsparse representation. TPAMI, 36(2), pp:261–275,(2014)[4] E. Elhamifar, G. Sapiro, A. Yang, S. Sasrty: A convex optimization framework for active learning.In: ICCV. pp. 209-216 (2013)[5] S.-J. Huang, R. Jin, and Z.-H. Zhou: Active learning by querying informative and representativeexamples. TPAMI, 36(10), pp: 1936-1949(2014)http://mulan.sourceforge.net/datasets-mlc.html